Pocket Forth, version 0.6.3 by Chris Heilman 7/4/93
(This is part 1)
Introduction
Forth is a beautiful language. It is easily spoken and source code looks like poetry. Oh yes, Forth produces fast compact code. Pocket Forth is a small Forth system for the Macintosh computer, founded on well documented principals(1,2,3). Although Forth is being standardized, Pocket Forth has evolved independently of the standard.
Pocket Forth is based on Flint by G. Yates Fletcher(4), figFORTH(5,6) and the Forth described in Starting FORTH by Leo Brodie(7). I wrote Pocket Forth to experiment with the Macintosh toolbox and Apple Events. Pocket Forth can produce applications that run under any system version.
Programs are easily modified and you are encouraged to adapt Pocket Forth into your own program. Toward this end, you own your copies of Pocket Forth. Programs may be distributed with no royalties, homage, or other restrictions, even if the Forth system is available to the end user.
NOTE — Pocket Forth is not warranted for any particular purpose. That means you assume all of the responsibilities in the use of Pocket Forth.
I have attempted to make Pocket Forth free of bugs, but low level languages, such as Forth, allow you access to the operating system. Therefore, use Pocket Forth with care in conjunction with other programs or files. This is especially true when writing programs that access the disk, or when experimenting while using other programs that do. Always open a backup of any file you may want to keep, including Pocket Forth itself. Never run originals!
If you are new to Forth, read Starting FORTH to learn the language. The Starting folder (inside of the Extensions folder) has more information on using Pocket Forth with Starting FORTH to learn the language.
Preparation
To use Pocket Forth, follow this procedure:
• Back up Pocket Forth before running it. Put the original away and make a working copy. Be sure to run the working copies only.
• Double click the program's icon. If everything is ok, a window will appear and say so. The 'ok' prompt and cursor indicate that Pocket Forth is ready for input.
Use other programming utilities, such as resource editors, to provide a rich programming environment.
About Pocket Forth
Pocket Forth has a simple interpreter that can execute code or compile it into a dictionary for later execution. Source code text is typed from the keyboard, pasted from the clipboard or loaded from a file. The text is interpreted one line at a time. When the line is completed an 'ok' prompt appears and the input sequence restarts.
The text interpreter parses tokens from the input line (also called the input stream). Tokens are any group of characters, excluding space, tab and carriage return. Individual tokens are separated from each other by one or more spaces. Carriage return signals the end of the line.
Each token of a line is looked up in a dictionary. The dictionary is a list of named subroutines, called words. Each of these words does a part of the program in a reusable subroutine. Words can be called by the interpreter or compiled into larger words.
In executing mode, the word's subroutine is run and the interpreter continues with the next token. Tokens not found in the dictionary are converted to a value. If the token is not a valid integer or floating point number, an error is signaled by printing a question mark and the unrecognized token.
Some words place the interpreter into compile mode. When in compile mode, new words are defined. Words are written into the dictionary as machine language subroutines. Thus, Pocket Forth produces subroutine threaded code. A program is built by factoring an algorithm into its parts, then defining words that assemble those parts into code. (12)
Here is the structure of the interpreter:
figure 1
Text Files
Typing into the Pocket Forth window supplies text to be interpreted. None of what is typed is saved, so write programs with an editor.
Pocket Forth does not use Forth's traditional block input, rather it interprets normal text files. This allows program source files to be prepared with your favorite text editor and tools. Which editor you choose will depend on memory available and personal preference.
On Macintoshes with MultiFinder or System 7, the Pocket Forth application runs in multitasking mode. A 64K byte segment is the minimum, but 128K bytes or can be used. This allows Pocket Forth to run simultaneously with an editor. Accessory editors can be used with Pocket Forth on even the smallest Macintosh.
Text from a file can be interpreted by Pocket Forth from the clipboard via copy and paste, or directly from the file. You may select and copy text from a source file and paste it into the Pocket Forth window. The pasted text is printed and interpreted just as if it had been typed in. The 'ok' prompt returns when the paste is complete.
Files are also interpreted directly. The basic file loading word is "-->" (pronounced "load"). Follow "-->" with the name of the file. If the file is in a folder, a path name must be used in the following manner:
--> diskName:folderName:fileName
This will load the file called 'fileName' in the folder 'folderName' from the disk 'diskName'. If the file to be loaded is in the default folder or on the top level of the startup disk, then only the file name is necessary. Launch a Pocket Forth from a folder to make it the default folder.
File, folder and disk names that are to be used with "-->" must not contain any spaces because the text interpreter stops at spaces, causing the entire name not to be found. This or other disk errors print an error message and return control to the keyboard. See the error section for more information.
The word "open" presents a standard file open dialog to peruse. Use it to select a file to interpret. There are no restrictions on the filename or location when "open" is used. The File menu has the Open… command available on it, which calls "open".
If System 7 is being used, files can be loaded by dragging the file icon to the Pocket Forth icon in the Finder. Release the file icon when Pocket Forth becomes selected. This causes Pocket Forth to start up if it is not already running and the file to be loaded.
Files should be saved from the editor prior to pasting or loading into Pocket Forth. Text files require a carriage return (ASCII: $0D) at the end of each line. Lines longer than 80 characters choke the interpreter.
A file stack is maintained so that files can load other files. Files can 'nest' up to five levels deep. When a file is finished interpreting, the input stream returns to the previous file and begins with the line after the "-->" was encountered. When all levels have been completed, control returns to the keyboard. If more than 5 files are nested, an error will result.
The text of the current file and all pending files is kept in memory. While files can be any size, the memory available will limit how large a file will load. Use short files that deal with single aspects of a program. Reusable code in small files can be loaded by other files. Simplify source code by loading a list of modular files and code that is specific to a particular program.
Comments document a program within the source code. Create comments by enclosing text within parenthesis or preceding the comment with a back—slash. Two main types of comments are stack comments and general comments, however their different structure is by convention only.
A stack comment is a one line diagram of the effect of a word's execution on the parameter stack. Stack comments may also show compile mode or the return stack. The purpose of a stack comment is to unambiguously show the effect of a word. See the stack section. A stack comment: ( before.parameter -- after.parameter ).
General comments tell what a word does, or why it has been defined. A short explanation comment should accompany each word defined. Long definitions may contain many comments. By commenting well, your program will be easier for others to understand and easier for you to maintain. See the example files for the use of both types of comments.
Memory
Pocket Forth uses memory in six ways. There are two stacks, variables, the dictionary, free memory available to the dictionary and absolute addresses.
Stack addressing is implied, that is, you push and pop, and the stack determines what address the actions access. Variable, dictionary, and free memory addressing are all done with a 16 bit relative address. Absolute addresses cover the entire addressing range of the 680x0 with a 32 bit double number.
Understanding the use of relative and absolute addressing is vital to programming Pocket Forth.
Absolute Addressing
An absolute address is the actual address as seen by the microprocessor. Pocket Forth treats absolute addresses as double numbers. A double number is two consecutive 16 bit numbers or a 32 bit number. The double number words (such as "2swap", "d+" and "2@") can manipulate absolute addresses. Absolute addressing allows access to any address whether 24 bit or 32 bit addressing mode is used.
In addition to double number words, Pocket Forth has some words to deal directly with absolute addresses. The words "l@", "dl@", "l!" and "dl!" read and write absolute addresses outside of the relative address range.
See the glossary for the specifics of using double number and absolute addressing words.
Relative Addressing
The addresses used within Pocket Forth are relative addresses, that is, 16 bit numbers, relative to a base address. Pocket Forth can be anywhere in memory and relative addresses will remain constant.
Relative addressing uses the 'indirect with displacement' addressing mode of the 680x0. Addresses within ±32K bytes of the base address can be accessed using a 16 bit relative address. Pocket Forth's dictionary starts at zero and uses only positive relative addresses, so the maximum dictionary size is 32K bytes.
Free memory, that is, memory available for dictionary expansion, is determined with the word "room". Pocket Forth is distributed with the maximum amount of free space available. The default amount can be changed by adjusting the 'freeSize' variable. Setting a size greater than 32 kbytes will cause a crash.
'Address' or the abbreviation 'addr' usually means a relative address. Relative and absolute addresses can be converted from one to the other. The word ">abs" converts a relative address to a double number absolute address.
The word ">rel" converts an absolute address to a relative address. Only absolute addresses within the relative addressing space will be correctly converted. Absolute addresses outside of this range are mapped into the relative addressing space, causing the wrong address to be calculated.
Stacks
The stack data structure is used throughout the computer world, but most languages hide their stacks. On the other hand, Forth stacks allow complete access.
Like most Forths, Pocket Forth uses a parameter stack and a return stack. The term 'the stack' used by itself usually means the parameter stack. In contrast, 'the stack' in other languages means the system stack, which is Pocket Forth's return stack.
The parameter stack, used to pass values to and from routines, is under your full control. It holds up to 1024 sixteen bit numbers or about 200 floating point numbers. See the error section about under- and overflow conditions.
figure 2
Because the return stack is the system stack, it is used for subroutine threading by Pocket Forth and other programs, including the operating system. Runtime threading by pushing and popping addresses to the return stack is handled automatically by the processor.
The return stack can also be used explicitly to store temporary data and to interface with toolbox routines. Used for data storage the return stack makes definitions smaller and faster. Here are two versions of a word that returns a product and a quotient:
In the first definition, "pandq" is written using parameter stack manipulation only. Holding n3 temporarily on the return stack makes the second definition smaller. It is also faster because return stack words, such as "r", "r>" and ">r", compile inline, as two byte macro definitions, instead of subroutine calls.
Used this way, the return stack must be carefully balanced between the beginning and end of a routine. An unbalanced parameter stack can cause an incorrect result, but a single number out of place on the return stack causes an immediate or delayed crash.
Toolbox parameters and results are passed by pushing and popping the return stack. In addition to the words ">r"and "r>", Pocket Forth adds the words "2>r", "2r>" and "a>r" to facilitate calling toolbox routines. Like the standard return stack words, these words must be balanced within a single colon definition. Refer to the glossary, the example programs and the toolbox section of this manual for further information on the use of these words.
Dictionary
Pocket Forth's dictionary is a linked list of subroutines, variables and other objects. Each object in the dictionary is named so that it may be matched to a token from the input stream. This makes the interpreter seem conversational.
A subroutine named "search" provides the ability to quickly locate any named routine. Given a token, "search" returns the address of the routine. Execution, compilation or some other action may be taken with the address.
Normally words that are typed (or loaded, or pasted) are executed immediately. The interpreter can also be in compile mode. The word ":" (pronounced "colon") engages the compiler to add a new definition to the dictionary.
NOTE: Actually the word "]" ("right bracket") enters compile mode by setting the user variable 'fcolon' to 128. "Colon" creates the dictionary header, that is, the name and link fields, then calls "]". Like "colon", ";" (pronounced "semi-colon") isn't the actual word that leaves compile mode. The word "[" ("left bracket") is called by "semi colon" to do that.
Anatomy of Dictionary Entries
"Colon" definitions consist of a name, a link address for searching and an executable subroutine. The subroutine contains code compiled by the interpreter. These definitions are themselves subroutines, therefore, definitions can call other definitions, nested as deep as the size of the return stack will allow.
This simple example program illustrates the dictionary's structure:
: HI ." HELLO" ; ( print "hello" on the screen )
: LINE hi 33 emit cr ; ( print hello! and go to next line )
: TEST BEGIN line ?terminal UNTIL ; ( continue 'til key press )
This figure shows the memory used in the example to illustrate how definitions are constructed:
figure 3
The three fields common to all definitions can be seen, as well as some special features in each one. Notice that dictionary memory is commonly used in 2 byte groups. These groups are called cells (or sometimes words, but that is confusing).
Name Field
The first part of a dictionary entry is its name field. A name field is four bytes long: the first three characters of the name are preceded by the number of characters in the name. Names such as "hi", with less than three characters, are padded with zeros so the name field is always four bytes long. Names with more than three characters are truncated.
Name collisions occur if the name field of two words is the same. For example, a word called "lint" would have the same name field as the word "line". The effect of a name collision is that references to the collided word are interpreted as the later defined word. References already compiled are executed normally. Redefining a word does not cause an error message because the dictionary is not searched during compilation, so choose names carefully. Many bugs can be traced to name collisions.
Link Field
After the name field is the link field. This two byte field holds the name address of the word defined just prior to the current word. "Search" uses the link address to scan through the dictionary from word to word. A dictionary search starts with the last word defined, whose name field address is returned by the word "latest".
Code Field
The code for a word begins at its code field address, usually referred to as simply 'the address' of the word. The address of a word can be found with the word " ' " (pronounced "tick"). Provided with the code address, the words ">name" and ">link" return the name and link addresses.
The code field consists of a subroutine, ending either with an RTS instruction or a jump to another subroutine. Most words compile to the form: JSR d(BP), where the displacement, d, is the relative address of the word.
Other features besides subroutine calls are found in the code field. Some words write code into the dictionary, or take some other action while in compile mode. These words are called immediate words, because they execute instead of compile when used within definitions. Some uses of immediate words are branching, literal numbers, strings and compiler control.
Immediate words used by Pocket Forth are identified in the glossary. To make a new definition immediate, follow the definition with the word "immediate". To compile an immediate word, precede it with the word "[compile]".
Strings, such as 'HELLO' in the example are compiled as ASCII characters prefaced with a length byte. The routine (.") prints the string data, then adjusts the return address to skip over the string. The immediate word "dot-quote" compiles (.") and a string into the dictionary, but does not print the string.
NOTE — Pocket Forth's "dot-quote" is not 'state smart'. Its behavior outside of a definition is the same as its compiling behavior, that is "dot-quote" compiles a call to (.") and the string data. To have a word that prints interactively, define "dot-paren":
: .( 41 word here count type ;
.( This is a test) ( will print the enclosed text )
Numbers, when used in a definition, compile inline code that pushes the value to the stack. Using the example from figure 3, in the word "line", the value 33 compiles as MOVE #33,-(PS) that is, push 33 to the parameter stack. The effect of this code is to put 33 on the stack when the code runs. Numbers entered into the interpreter are pushed to the stack, like an RPN calculator.
The word "literal" compiles a value as described above (known as a literal value) from data on the stack rather than from the input stream. The word "dliteral" works like "literal" but compiles a 32 bit push instruction from a double number. Floating point numbers work like integers but occupy ten bytes on the stack. "Fliteral" compiles five consecutive push instructions.
Branching is controlled by immediate words that place short test and branch routines into the code. These branching structures are supported by Pocket Forth:
BEGIN ... (flag) UNTIL loop until flag is true
BEGIN ... (flag) WHILE ... REPEAT Two part loop until flag is false
BEGIN ... AGAIN loop forever ...
(flag) IF ... THEN do enclosed code if flag is true
(flag) IF ... ELSE ... THEN do 1st part if true, 2nd if false
(limit start) DO ... LEAVE ... LOOP do code (limit-start) times
(limit start) DO ... LEAVE ... (count) +LOOP do code ((limit-start)/count) times
("leave" is optional)
Between "do" and "loop" the 16 bit value on the return stack, returned by "r" is the index of the loop. You can test or change the value of "r" within a loop. "Leave" sets the index to the limit; this causes the loop to exit at the next occurrence of "loop" or "+loop".
Branching words must be used in the correct combination; no compile time error will occur if they are not, but a crash is likely to at runtime.
NOTE: Back branches, that is branches toward lower memory are done by the word "back". "Back" will compile short branches if the displacement is less than 128 bytes, otherwise, long branches (four bytes) and compiled. Back is called by "until", "repeat", "again", "loop" and "+loop" so you usually don't need to call it yourself.
Macro Definitions
Another class of definition, similar to immediate words, are macro definitions. A compiled subroutine is four bytes long, and the JSR/RTS combination has a small, but noticeable time penalty. Routines that are only two or four bytes long are defined as macros. Macros compile their code inline rather than as subroutines. Inline words are the same size or smaller and execute a faster than subroutine words (2,8). When not in compile mode, macro words function as subroutines.
These words are defined as 2 byte macro definitions:
!PEN CBLK OR + CSTATE OVER +MD D+ PMODE -TO DL! R0@ 2OVER DL@ S0@ AND L! TIB BASE L@ XOR FDROP INTERPRET
You may also define your own macros. Declare a word to be a macro definition by following the word's definition with the word "macro". A major caveat applies to user defined macros. Definitions become very large if macros longer than four bytes are used. This means that macros should be machine code only. If speed or maintaining the return stack is a concern, macros may be made from code longer than four bytes. See the Sieve file for an example of macros built for speed.
Here are some useful macro definitions:
: 00>R ( rstack: -- 0 0 ) ( push a long zero to the return stack )
,$ 42A7 ; macro \ clr.l -(rs)
: 2R ( -- d ) ( rstack: d -- d ) ( get double from the return stack )
,$ 2D17 ; macro \ move.l (rs),-(ps)
Parameter Field
The parameter field is present in words defined using "variable", "create" or other 'defining' words. Defining words build new words, called 'daughter' words, using the next token from the input stream for the name. When the daughter word later executes, it returns the address of its parameter field to the stack. If the "does>" follows "create" in a definition, the daughter word also executes the code after "does>".
In a definition, words that follow "create" build a data structure during compilation. Words after "does>" operate on the data at runtime. Here is an example:
Variables are defined with the word "variable", "2variable", "fvariable" or "create". Use "variable" to create data structures if "does>" is not needed. Variables are smaller and faster than 'created' data structures. If larger storage areas are needed than "variable" or "2variable" provide, use "allot" which adds bytes to the parameter field.
Memory Management
In a Macintosh, certain conditions cause functional blocks of memory to move around(9). This is called memory management and is done by the memory manager in the ROM. (Brodie's fans should imagine a cartoon character for the manager.) Because of this, Macintosh programs must run from any memory location.
Memory available for a new program is allocated above the dictionary. Do not confuse this with addressable memory. Allocated memory is memory you are allowed to use by the memory manager. Addressable memory can be accessed with a relative address. Programs that alter memory should respect the limit of the allocated memory when using relative addressing. Pocket Forth comes with 32K (the maximum) allocated.
The dictionary is loaded from the disk as a 'DICT' resource. The entire dictionary can be written back to the disk with the word "save", replacing the previous DICT resource. This makes extensions and modifications a permanent part of your copy of Pocket Forth. NOTE: "save" cannot be undone.
The memory map, with high memory at the top:
figure 5
Other programs that may be in memory are not shown in this diagram.
Errors
Three types of error are reported: stack errors, numeric conversion errors and disk errors. All of these errors use the general purpose error word "abort". "Abort" prints a question mark, clears the file, parameter and return stacks and goes back to the prompt. A fourth type of error is not reported, the Apple Event error. See the Apple Event section for more details.
Stack errors occur when items are removed from the parameter stack when it is already empty. Since the parameter stack is often 'under flowed' this way, it is buffered so that no harm will be done by this condition. A stack error report looks like this: *?. Make a stack error happen by typing "drop" when you know the stack is empty.
Stack overflow on the other hand is not reported nor buffered. A runaway loop is the most often cause of stack overflow. The symptoms differ in each occurrence from a bomb box to extreme graphics and rude noises. Check your loops carefully.
Tokens not found in the dictionary are assumed to be numbers. If the conversion routine, "number", finds any characters greater than the value of "base", the token is not an integer either. Then the interpreter uses SANE to determine if the token is a valid floating point number. Usually these are numerals with a decimal point or an 'E', however, any undefined token beginning with a number is converted. Unconverted tokens are reported by printing the token in capital letters, then doing "abort". Type "xx" to see this error.
Disk errors are reported by printing "Disk:" and the number of the error encountered. Look up the error numbers in Inside Macintosh(10) or other text. "Abort" is executed after the message is printed.
Common disk errors include:
-35 No such volume error: disk name is misspelled or disk is not mounted.
-36 General purpose I/O error: cause unknown.
-43 File not found: pathname is misspelled or includes a space.
Use "abort" to write error handlers. Type a message then call "abort" as in this example:
: BASIC ( -- ) here 20 expect ( get a line of text )
." Syntax Error" abort ( report error and abort )
beep ; ( NOTE: this never happens )
Errors reported by some other Forth systems are not reported by Pocket Forth. Compiler or branching security is not provided, nor is free space checked before compiling. Multiple definitions of the same word are allowed. The lack of these messages can sometimes cause mysterious problems.
If the dictionary runs out of free space while compiling, compilation continues on, right into whatever is right after the DICT block in the heap. This is likely to cause a crash, so you must be aware of the memory available for your program. Remember to check free dictionary space with "room".
(The manual is continued in the file Manual(part2).)
